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FAULT DETECTION & DIAGNOSIS USING NEURAL NETWORK 

APPROACHES 

Prof. Mark A. Kramer 


ABSTRACT 

Neural networks can be used to detect and identify abnormalities real-time process data. 
Two basic approaches can be used, the first based on training networks using data representing 
both normal and abnormal modes of process behavior, and the second based on statistical 
characterization of the normal mode only. Given data representative of process faults, radial 
basis function networks can effectively identify failures. This approach is often limited by the 
lack of fault data, but can be facilitated by process simulation. The second approach employs 
elliptical and radial basis function neural networks and other models to learn the statistical 
distributions of process observables under normal conditions. Analytical models of failure 
modes can then applied in combination with the neural network models to identify faults. 
Special methods can be applied to compensate for sensor failures, to produce real-time 
estimation of missing or failed sensors based on the correlations codified in the neural network. 
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How to use neural networks to: 


1) detect 

2) identify 

3) rectify 

faults in processes and associated sensors. 


Using neural nets involves learning from 
examples. 

Two approaches: 

1) Learning with examples of normal and 
abnormal behavior 

2) Learning with examples of normal 
behavior only 

Useful in the absence of a functional theory 
of device behavior. 
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RFVIFW OF NEURAL NETWORKS 


Artifical neural networks are loosely based on how 
the brain carries out its low-level computations. 

• Simple computational elements acting in parallel 

• Neurons "fire" when excited by other neurons 

• Capable of learning and responding differently to 
different input patterns 



Input/output behavior determined by: 

• Topology of network 

• Computation of each neuron 

• Adjustable parameters of connections and nodes 
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Two of the most important types of networks are: 

• backpropagation networks (BPNs), 

• radial basis function networks (RBFNs) 




Computation in Backpropaaation Networks 


• Layered architecture, usually input layer, output layer, plus 
one intermediate "hidden" layer 

• Connections between nodes are weighted. Weights 
multiply the signal on the connection. 

• Each node sums its inputs and then passes the result 
through a sigmoidal nonlinearity 



Typcial sigmoid function: f(u) = 1/(1 +exp(-u)) 
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Computation in Radial Basis Function Networks 

• Similar to BPN but uses Gaussian nonlinearity in nodes 

• Input/hidden layer connections not weighted, simply 
pass input vector X to hidden layer 


hidden layer 



units 


Each Gaussian unit has internal parameters representing 
a "unit center" m and a "receptive width" a 

Output of Gaussian unit i, aj, is based on the distance 
between inputs x and the unit center rm 

/ - x - rrij \ 
aj =exp( M - g ~U Lj 
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Graphical interpretation of hidden nodes of RBFN 

One input dimension: 



Multiple dimensions: 

data 
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NFIIRAL NETWORK LEARNING 

Takes place through an optimization of internal 
connection weights W. 

A set of examples of desired input/output behavior 
(x, y)j, i=1 ,—K is required. 

A least-squares fit is sought: 


K 



APPLICATION PHASE 

INPUT VECTOR X OUTPUT VECTOR Y 
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Y = Y(X,W) 





Multi-Class Diagnosis Using Neural Networks 



Neural network is capable of identifying 
more complex class regions than "high- 
low-normal"-style rules 


sensor 1 
sensor 2 
sensor 3 



Fault 1 
Fault 2 
Normal 


NEURAL NETWORK 
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Network Training For Multiclass Diagnosis, 


observables x diagnosis y 

(0.032, 0.099, -0.039} -» fault 1 { 1, 0, 0) 

(0.016, -0.53,-0.465} -> fault 2 (0,1,0} 
{0.466,0.022,-0.405} -» fault 3 -» {0,0,1} 


Example inputs x: 

Feedstock characterization: (sp. grav., bubble pt., vise.,..} 
Sensor data: (meas 1 , meas 2,....} 

Time series: (meas(t), meas(t-l), meas(t-2), ...} 


Using least squares objective function, assuming: 

1 ) Sufficient # of training examples 

2) Examples in proportion to prior probabilities 

3) Adequate network representational capacity 


Then: 


P(fault i | x) 

yi= £P(fauitj|x) 

j 


relative 
= probability 
of fault i 


£y; * 1 implies an invalid classification 
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(See Kramer & Leonard, Comput. Chem. Eng., 14, 1323, 1990) 



RADIAL BASIS FUNCTION NETWORKS ARE 
BETTER FOR DIAGNOSTIC PROBLEMS THAN 
BACKPROPAGATION (SIGMOIDAL) NETS 
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Fault diagnosis example problem 

(see Kramer & Leonard, IEEE Control Systems 1_L 31 , April 1 991 ) 

3 classes, 2 input dimensions 
30 training examples of each class. 



Legend 

□ C 1 (Normal) 
♦ C 2 (Fault 1) 
■ C 3 (Fault 2) 
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Backpropagation networks (sigmoidal nodes): 

• Class regions divided with hyperplanes 

• Tends to place class boundaries near "edge" of class 

• Check sum XYj * 1 indicates some regions of 
insufficient training data (sufficient but not necessary) 




Legend 

□ C 1 (Normal) 
♦ C 2 (Fault 1) 
■ C 3 (Fault 2) 
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DECISION REGIONS 






Radial Basis Function Networks (Gaussian nodes): 

• Well-placed classification boundaries 

• Check sum XYj = 1 tends to be satisfied everywhere 
(i.e. no regions flagged as novel) 




Legend 

□ C 1 (Normal) 
♦ C 2 (Fault 1) 
■ C3 (Fault 2) 
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DECISION REGIONS 



Closer look at novelty in RBFNs: 

. Radial units are centered among groups of data 
by k-means clustering. 

• Gaussian activation functions a(x) decrease to 0 
as one moves away from unit center 


a,->0 



• Novelty can be indicated by max(aj) < cutoff value 
(e.g. 0.5) 

• Works even better with elliptical units 
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Flliptical Basis Function Networks (EBFN 


• Similar to RBFN but unit shapes can be 
elliptical 

• Shapes determined by local covariance 
structure of data 

• Good novelty detection and classification 
properties 



Elliptical Basis Function Coverage of 
Fault Classification Data 
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Data Density Estimation using RBFNs 


In each radial or elliptical unit, local data density is 
approximately: 

_ # data points local to unit h 

h “ Volume of unit h • total # data points 


A smooth data density estimate at every point in 
space then given by the interpolation formula: 



fi 3h(x) Ph 

h=3 

1 - max(a h ) + 2 3h(^) 

h=1 


Uses of probability density function: 

1) Class-based decomposition of classifier 

2) Fault detection using only normal data 

3) Rectification of sensor faults 


(See Leonard, Kramer & Ungar, Comput Chem Eng, vol. 16,819, 1 992) 



Estimated density 


sensor 2 
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CLASS-SPECIFIC DENSITY ESTIMATION 



Class-Based Network Decomposition 


Pj(x) = P(x | Hj) 

Density related to posterior fault probability 
conditional on data via Bayes’ Theorem: 

P(Hj | x) = P(x | P(Hj) / P(x) 

P(x) is pooled density function (all classes). 
Relative probability, eliminate P(x): 

R ik = P(x | H,) P(Hj) / P(x | H k ) P(H k ) 

Class-decomposed network: 


Xi 

x 2 




d 

1 


s 

1 

s 
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O 3QTQ P 
















Decomposition benefits: 

• No regression of weights 

Work savings = 0(HM/N) 

H = # hidden nodes 
M = # faults 
N = # inputs 

• Allows incremental development of classifier, 
easy incorporation of new data 

• Prior probabilities and misclassification costs 
can be incorporated in Bayesian decision 
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FAI II T DETECTION USING DENSITY FUNCTION 


Fault detection: Is current state in the normal class, 
or out? 

Fault data not needed. 

• Model probability density function of normal class 

• Place probability limits, e.g. 95% for declaring fault 
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Rectification of sens or faults by 
probability optimization 


raw sensor 
readings 


Rectification 


estimated states 


sensor errors 


Assume model of normal probability distribution 
Hypothesize sensor failures only 


X2 


# recorded sensor value, y 
8 , adjustment = y - x 

° o f 

° c f> rectified state, x 



normal data distribution 



maximize P(x|y) <*= P(5)P(x) 



likelihood of likelihood of 
adjustment rectified state 




Example: Plate temperature rectification in distillation 



normal 

operation 

probability 



5 0 initial 

conditions 
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I-Sl Sc Meas T2 Est Sc Mcas T1 


Test: Corrupt each of 5 sensors in test set of 100, 
yielding 500 examples with single sensor failure. 


Rectification for Temperature 1 



Rectification for Temperature 2 
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Est & Meas T5 


Rectification for Temperature 5 



True Temperature 5 




Summary & Conclusions: 

Diagnosis can be approached by: 

• Multi-class training 

• Single-class training 

Multi-class diagnosis yields relative fault 
probabilities 

Radial basis function networks preferred 
approach to multi-class diagnosis 


Single-class training involves extraction of 
statistical distribution model 

Can be approach using radial basis functions 

Useful for: 

• Decomposing multi-class problems 

• Fault detection 

• Rectification of sensor data 
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